Skip to content

feat: make cluster domain configurable in KEDA deployer#3682

Open
Itx-Psycho0 wants to merge 1 commit into
knative:mainfrom
Itx-Psycho0:fix/3403-configurable-cluster-domain
Open

feat: make cluster domain configurable in KEDA deployer#3682
Itx-Psycho0 wants to merge 1 commit into
knative:mainfrom
Itx-Psycho0:fix/3403-configurable-cluster-domain

Conversation

@Itx-Psycho0
Copy link
Copy Markdown
Contributor

Problem

The external name service to the keda-add-ons-http-interceptor-proxy was hardcoded with the cluster domain set to cluster.local. This works for most cases, but fails in clusters with custom domain configurations.

Solution

Made the cluster domain configurable by adding a ClusterDomain field to the DeploySpec struct. The KEDA deployer now uses this field to build the interceptor proxy service URL, with cluster.local as the default fallback for backward compatibility.

Changes

  • Added ClusterDomain field to DeploySpec in pkg/functions/function.go
  • Updated interceptorBridgeService() in pkg/keda/deployer.go to use the configurable cluster domain
  • Added unit tests in pkg/keda/deployer_test.go to verify the functionality

Usage

Users can now configure the cluster domain in their func.yaml:

deploy:
  deployer: keda
  clusterDomain: "custom.domain"  # Optional, defaults to "cluster.local"

Testing
✅ All unit tests pass
✅ Linting checks pass
✅ Backward compatible (defaults to "cluster.local")
Fixes #3403

This commit addresses issue knative#3403 by making the cluster domain
configurable for KEDA deployments.

Changes:
- Added ClusterDomain field to DeploySpec in pkg/functions/function.go
- Updated KEDA deployer to use configurable cluster domain with
  'cluster.local' as the default fallback
- Added unit tests to verify the cluster domain configuration works
  correctly with default, custom, and explicit values

The external name service to keda-add-ons-http-interceptor-proxy now
uses the configured cluster domain instead of being hardcoded to
'cluster.local'. This allows the function to work in clusters with
custom domain configurations.

Fixes knative#3403
@knative-prow
Copy link
Copy Markdown

knative-prow Bot commented May 8, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Itx-Psycho0
Once this PR has been reviewed and has the lgtm label, please assign dprotaso for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow knative-prow Bot requested review from dsimansk and jrangelramos May 8, 2026 17:31
@knative-prow knative-prow Bot added size/L 🤖 PR changes 100-499 lines, ignoring generated files. needs-ok-to-test 🤖 Needs an org member to approve testing labels May 8, 2026
@knative-prow
Copy link
Copy Markdown

knative-prow Bot commented May 8, 2026

Hi @Itx-Psycho0. Thanks for your PR.

I'm waiting for a knative member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@Itx-Psycho0
Copy link
Copy Markdown
Contributor Author

Hi maintainers,

I've implemented a fix for issue #3403 that makes the cluster domain configurable in the KEDA deployer.

Summary:

  • Added a ClusterDomain field to DeploySpec that allows users to configure the cluster domain
  • Updated the KEDA deployer to use this field when building the interceptor proxy service URL
  • Defaults to "cluster.local" for backward compatibility
  • Added comprehensive unit tests

The implementation is backward compatible and follows the existing patterns in the codebase. All tests pass and linting is clean.

I'd appreciate your review when you have a chance.

@lkingland @creydr @matejvasek

Thanks!

@matejvasek matejvasek requested review from creydr, gauron99 and lkingland and removed request for dsimansk and jrangelramos May 13, 2026 12:30
@matejvasek
Copy link
Copy Markdown
Contributor

Is this only keda specific?

@matejvasek
Copy link
Copy Markdown
Contributor

@creydr please look at this.

@matejvasek
Copy link
Copy Markdown
Contributor

Isn't cluster domain setting of a cluster, not a function?

@gauron99
Copy link
Copy Markdown
Contributor

Is this only keda specific?

wouldnt this also touch on raw k8s deployer?

@Itx-Psycho0
Copy link
Copy Markdown
Contributor Author

You're absolutely right on all points. Let me address the concerns:

"Is this only KEDA specific?"
No, it's not. The cluster domain affects any deployer that builds internal service URLs. The raw K8s deployer likely has similar hardcoded references to cluster.local.

"Isn't cluster domain setting of a cluster, not a function?"
You're correct, the cluster domain is a cluster-level configuration, not a function-level one. Putting it in DeploySpec (per-function) is architecturally wrong.

Proposed Better Solution - Auto-detect cluster domain from the cluster's DNS configuration
Make it a global/cluster-level config (not per-function)
Apply to all deployers (KEDA, raw K8s, Knative if needed)

Implementation approach:
Add cluster domain detection in the deployer initialization
Fall back to "cluster.local" if detection fails
Allow override via global config (not per-function)
Apply consistently across all deployers

Should I rework the PR with this approach? I can:
Move cluster domain to a global config
Add auto-detection logic
Apply it to both KEDA and raw K8s deployers
Add tests for all deployers

Let me know if this direction makes sense!
@gauron99 @matejvasek

@lkingland lkingland added ok-to-test 🤖 Non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test 🤖 Needs an org member to approve testing labels May 16, 2026
@lkingland
Copy link
Copy Markdown
Member

lkingland commented May 16, 2026

This one is... nontrivial.

However, if you're interested in tackling it (with @creydr's help I hope, nudge nudge) your other PRs have been quite nice, so I am sure we can figure out a good solution.

I've added comments to the parent issue #3403

@lkingland
Copy link
Copy Markdown
Member

lkingland commented May 16, 2026

Here, I had AI do some research and break it down into actionable steps. Beware potential hallucinations may follow:

Shared contract (decide first — blocks everything else)

  • Pick the ConfigMap coordinates. Propose func-system/cluster-config; agree namespace + name as a stable client/server contract.
  • Define the schema. At minimum clusterDomain. Reserve keys for defaultRegistry, installedComponents, supportedRuntimes, defaultNamespace. Document forward-compat
    rules (unknown keys ignored, never repurpose a key).
  • Decide kubeconfig-context as the cluster identity key on the CLI side (used for cache directory and user-override scoping).

Operator (depends on contract)

  • Add cluster-domain detection that parses /etc/resolv.conf search entries in the operator pod (port of Knative's GetClusterDomainName logic).
  • Reconcile the cluster-config ConfigMap on operator startup, creating/owning it via metadata.ownerReferences to the operator Deployment so it cleans up on
    uninstall.
  • Add a periodic reconciliation timer (e.g., 1h) so post-install kubelet reconfigs eventually propagate.
  • Emit a ConfigMap event on change so consumers can watch instead of poll if they want to.
  • Document the ConfigMap as the operator's public API surface in the operator README.

CLI core helper

  • Introduce a single func ClusterDomain(...) helper that every deployer goes through. No deployer should reference cluster.local directly.
  • Implement the precedence chain inside the helper:
    a. --cluster-domain flag value (if set on the command)
    b. ~/.config/func user override for the active kubeconfig context
    c. func-system/cluster-config ConfigMap read from the cluster
    d. cluster.local
  • Add CLI cache at ~/.cache/func//cluster.json with a short TTL (default ~1h, env override). Invalidate when kubeconfig context changes.
  • Add --cluster-domain flag to func deploy (and any other commands that build cluster-internal URLs) as the explicit override.
  • Improve the fallback error UX: when a deploy fails on the cluster.local default with no operator and no user override, the error message should say "assumed
    cluster.local; set X to override" — not silent.

Global config (~/.config/func)

  • Add a clusters: section keyed by kubeconfig context name, each with a clusterDomain field (and reserved space for the same future keys as the cluster ConfigMap).
  • Define a single source of truth for config schema so the in-cluster ConfigMap and the user-local file share field names.
  • Add a func config cluster set clusterDomain subcommand (or similar) so users don't have to hand-edit YAML.

Deployers (consume — depends on CLI helper)

  • pkg/keda/deployer.go:231 — replace hardcoded ExternalName with ClusterDomain(...) lookup.
  • pkg/k8s/deployer.go:322 — replace hardcoded Trigger filter Host with same.
  • pkg/pipelines/tekton/templates.go — audit and replace any cluster.local literals (verify which need it; some Tekton templates may run inside the cluster and could
    use env-var injection instead).
  • Grep for any remaining cluster.local literals in pkg/ and cmd/ and either route through the helper or annotate why they're intentional (e.g., test fixtures).

Docs

  • New reference page: "Cluster Configuration" — explains the operator ConfigMap, the user-local override, the flag, and the precedence order.
  • Troubleshooting entry for the symptom (KEDA interceptor unreachable / Trigger filter 404s) pointing at the override.
  • Note in operator README that the cluster-config ConfigMap is part of the operator's contract.

Suggested PR sequence

  1. Contract + helper skeleton (no behavior change yet — helper just returns cluster.local)
  2. Deployers wired through the helper (still cluster.local, but centralized)
  3. Global config schema + user override path
  4. Operator side (detection + ConfigMap publish)
  5. CLI ConfigMap reader + cache
  6. Docs + troubleshooting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test 🤖 Non-member PR verified by an org member that is safe to test. size/L 🤖 PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Clusterdomain is not configurable in Keda deployer

4 participants